Fix: Prevent flaky tests with random values in parametrize#3
Merged
MKaczkow merged 2 commits intoMKaczkow:masterfrom Apr 26, 2026
Merged
Fix: Prevent flaky tests with random values in parametrize#3MKaczkow merged 2 commits intoMKaczkow:masterfrom
MKaczkow merged 2 commits intoMKaczkow:masterfrom
Conversation
📝 WalkthroughWalkthroughA pytest fixture was added to the test file that seeds Python's Changes
Estimated code review effort🎯 1 (Trivial) | ⏱️ ~2 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
testing/pytest/test_employee_pytest.py (1)
33-60:⚠️ Potential issue | 🟠 MajorSeed before parametrization, not in a fixture.
pytestfixture bodies run after collection, but therandom.randint()calls on lines 52–60 are evaluated when the decorator arguments are built during module import. This fixture does not make these parameters deterministic.Proposed fix: use a local seeded RNG for parametrized values
-# fixture to ensure reproducible random values -# without seed, random.randint() produces different values each run (flaky tests) -@pytest.fixture(scope="session", autouse=True) -def set_random_seed(): - """make tests reproducible by seeding the random number generator.""" - random.seed(42) - yield +_random_values = random.Random(42) +_random_test_cases = [ + (_random_values.randint(0, 100),) * 2 + for _ in range(9) +] @@ - tuple([random.randint(0, 100)] * 2), - tuple([random.randint(0, 100)] * 2), - tuple([random.randint(0, 100)] * 2), - tuple([random.randint(0, 100)] * 2), - tuple([random.randint(0, 100)] * 2), - tuple([random.randint(0, 100)] * 2), - tuple([random.randint(0, 100)] * 2), - tuple([random.randint(0, 100)] * 2), - tuple([random.randint(0, 100)] * 2), + *_random_test_cases,🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@testing/pytest/test_employee_pytest.py` around lines 33 - 60, The parametrized random values are generated at import time so the set_random_seed fixture doesn't make them deterministic; replace those random.randint(...) calls in the pytest.mark.parametrize argument with values produced from a local seeded RNG (e.g. create seeded_rng = random.Random(42) at module scope just above the pytest.mark.parametrize and call seeded_rng.randint(0, 100) for each tuple) and keep the set_random_seed fixture as-is; this ensures deterministic params while keeping set_random_seed for other tests, and refer to the existing set_random_seed fixture and the pytest.mark.parametrize block when making the change.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Outside diff comments:
In `@testing/pytest/test_employee_pytest.py`:
- Around line 33-60: The parametrized random values are generated at import time
so the set_random_seed fixture doesn't make them deterministic; replace those
random.randint(...) calls in the pytest.mark.parametrize argument with values
produced from a local seeded RNG (e.g. create seeded_rng = random.Random(42) at
module scope just above the pytest.mark.parametrize and call
seeded_rng.randint(0, 100) for each tuple) and keep the set_random_seed fixture
as-is; this ensures deterministic params while keeping set_random_seed for other
tests, and refer to the existing set_random_seed fixture and the
pytest.mark.parametrize block when making the change.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: fb7b33e4-850c-483a-a1e0-005c4ed7b347
📒 Files selected for processing (1)
testing/pytest/test_employee_pytest.py
Owner
|
Yep, thanks for the fix. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hey! This PR is part of a research project on Python projects.
I noticed this repository focuses on learning Python concepts, so I wanted to help highlight a common issue in tests.
The test
test_very_simpleuses random values inparametrize. Without a fixed seed, this can lead to flaky tests (different numbers on each run), which makes debugging harder.I understand that in this specific case nothing will break, since the test has only a simple assertion. Still, Im proposing this change as a best practice and as a learning opportunity.
This PR adds a fixture to set a random seed, making tests deterministic while preserving the idea of testing with multiple values.
Summary by CodeRabbit
Release Notes